Technical Field
The present invention generally relates to digital frame processing and in particular to panoramic frame processing.
It finds application in particular, while not exclusively, in picture acquisition devices and in particular, while not exclusively, in single lens devices.
Related Art
The approaches described in this section could be pursued, but are not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated herein, the approaches described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
Panoramic pictures or mosaics are larger images formed by assembling smaller still or video frames according to a single given direction (e.g. left to right) imposed by the movement of the camera that captured the sequence of frames. The sequence of frames can be acquired by a camera that is rotated (panned for example) around a given axis and in a given sense (for example clockwise or counter clockwise). For the sake of simplification, we consider that the camera is panned in a main direction.
Typically, a current frame is superimposed on the preceding frame in the sequence. Hence, a mix area of the current image typically corresponds to a common part between the current and one or more of the preceding frames. Also, a copy area of the current frame corresponds to a part of the current frame that is not common with any of the preceding frames and will complete the preceding frame for forming the panoramic picture.
The definition of the mix and copy areas thus comprises dividing the current frame into two parts (the mix area being the left part and the copy area being the right part, in case of a left to right movement of the camera).
The superimposition of the current frame on the preceding frame is performed by analysis of all the pixels of each frame, or by analysis of the movement of the camera.
Once the current frame has been superimposed on the preceding frame, the pixels of the current frame in the mix area are typically mixed or blended with the pixels of the preceding frame that are in the mix area of the current frame, and the copy area is added to the preceding frame to complete the panoramic picture comprising the previous frame.
Thus, a mosaic or a panoramic picture is obtained from a plurality of frames obtained from different point of views, as the camera acquiring the frames is panned.
This method can also be used to create stereoscopic panoramic pictures from a single video. This is done by generating at least two panoramic pictures, which can for example correspond to left and right views with a different positioning of the mix and copy areas.
As the camera is panned, the perspective is different in each frame of the recorded video. When a single panoramic picture is computed, the perspective differences are not a problem as the perspective differences between successive frames can be small enough to be mapped on a planar manifold. However, the different perspectives are problematic when several panoramic pictures are created, for stereoscopic panoramic pictures for example. Indeed, a given copy area for the left view in a first frame corresponds to a copy area for the right view in a second frame, which has a perspective that significantly differs from the perspective of the first frame. In addition to the perspective differences due to the rotation of the camera and which is present even in an ideal case, other perspective differences can be due to the fact that the camera motion is unrestricted in the case of a hand-held camera, and therefore that pitch and roll (rotations around the other axes) of the camera can vary significantly between each frame.
The result of perspective differences is that a final stereoscopic image, obtained based on the two panoramic pictures, will look distorted, unnatural, and the depth effect may be difficult to perceive, thus creating eye fatigue for a viewer.
Thus, there is a need to enable computing a non distorted stereoscopic image from at least two panoramic pictures acquired with a single frame acquisition device.